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Introduction 

Tlie tradition in the area of instructional technology has been consistent in its approach to 
research and evaluation. Tlic focus has been on documenting the success and failure of objectives. 
To a large extent, qualitative dimensions have been avoided. Unfortunately, in numerous research 
and evaluative efforts (especially those that are internally generated), objectives are not always 
clearly defined. Even those that are clearly defined in the proposal often suffer Murphy's law. 
Panicularly when a program is in its fixst year of operation, reductionistic and objectives-based 
data collection strategies generate insufficient data for ongoing program improvement. 
Increasingly, instructional designers and educational psychologists are turning to innovative 
melliods to collect and analyze data for interpretive research studies as well as naturalistic or 
improvement-oriented evaluations. 

For these reasons, we propose that one other data collection methodology be added to the 
"standard fare". Photo-interviewing (Tucker & Dempsey, 1990) was selected as the strategy 
because it offers much promise in collecting holistic and consistent samples or records of critical 
phenomena. The viability of this metiiod has been well documented in antiiropology and sociology 
(e.g., Becker. 1978; Byers, 1964; Collier & Collier, "1986; Garflnkel, 1967) and has been funher 
illuminated by Susan Sontag's (1977) cautions. Even so, photography as a method of inquiry has 
not been applied systematically in educational research or program evaluation (Fang, 1985; 
English, 1988). We need to know more about tiie rhetoric of words and pictures in educational 
inquiry. To unlock the meaning of tiiis information, one must examine more than empirical facts. 
We need to grapple with tiie problem of gathering subjective and intersubjective perceptions. 
Central to our investigation is die study of what Erickson & Mohatt (1982) call "immediate 
environments of learning", namely an analysis of how instructional contexts are socially and 
psychologically constituted. 

According to Collier & Collier (1986). "die production of photographs for purposes of 
research requires ...an overview of the cultiure or setting which allows all details to be seen within 
' a context." This whole view of a context can be achieved in a variety of ways. This paper will 
focus on some of die techniques and challenges of using photographic evidence and photo- 
interviewing in educational evaluation and research contexts. For die purpose of this paper, we will 
differentiate the tv.'o terms on the basis of five dimensions: purposes, role of die client, methods, 
controls, and ends. Research aims for generalization, gathering descriptions, seeking conclusions 
and trying to explain causation. Directed by the researcher, research's metiiods often emphasizes 
die empirical, seeks internal and external validity as its controls, and the ultimate results may or 
may not have an immediate payoff. Contrast Uiese characteristics widi evaluation. Evaluation aims 
for satisfying specific situations and synthesizes judgments from a myriad of data sources in order 
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to facilitai*; decisions, rather than explaining causes. Directed by the client, evaluation's methods 
arc multi-disciplinary, and uses utilitarian versus statistical controls (e.g., accuracy, utility, 
feasibility, and credibility). The ultimate results are expected to be immediate and practical. 

Why Photo-Interviewing ? 

We believe that photo-interviewing yields richer data than that usually obtained from verbal 
interviewing procedures alone. Informants tend to examine images and react to cues present in 
those images more carefully than would have been expected using written or spoken cues alone. 
Photographs trigger recall and focus the interviewing process, enabling an in-depth look at 
intended as well as unintended aspects of a program. Photographic images solicit both differences 
and similarities in individual perception which can be validated and analyzed across groups of 
perceivers. 

Photo-interviews can generate data particularly helpful for pilot-stage projects where 
innovation and development rather than status quo and maintenance are organizational priorities. 
Moreover, we posit that photo-interviewing can serve as an instructional as well as evaluative 
strategy given tiie projective nature of visual images. Photographs prompt reflection upon the 
program being evaluated which goes far beyond an interviewing simation lacking visual cues. 

Photographic evidence of behavioral patterns can be "detected" in many ways. In a 
classroom environment, for example, visual data can include student body language, seating 
patterns, and off-task behavior. Photographs appear to act both as stimuli and verifiers of 
perception. By providing a common stimulus for informants, photographic images both solicit 
differences and similarities in individual perception which can be analyzed across groups of 
perceivers. 

Moreover, a particularly potent argument for using photography (including film or video) is 
the potential for multiple applications in evaluation and research situations. Photographs capture 
direct observations of an environment and can serve as stimuli for interviews which in turn 
generate more data. In this sense, photography and its related media are particularly versatile in 
tiiose environments in which methodological triangulation is applicable. This argument also raises 
the valid counterargument that using photographs as data and stimuli, for example, may lead to . 
inconsistent or contradictory conclusions. This really is the argument against using any form of 
triangulation. Mathison (1988) asserts that it is unrealistic to expect that the use of multiple data 
sources, investigato'^, or methods can lead to a singular proposition about die phenomenon being 
studied. While his rationale may work from a behaviorist tradition, cognitivist models of reality 
seek holistic data collection, risking the price of inconsistent or even contradictory data in tiieir 
quest for the wealth of information derived trom multi-modality data sources. Admitting the 
inconsistencies and contradictions which must arise in using photo-related data sources is not 
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meant to justify an absence of some theoretical perspective. Rather, we contend that it is only 
through the use of robust cognitive theories that a researcher or evaluator has hopes of constructing 
plausible explanations of diverse phenomena, 
episodes: Ap plications in education and training 

Our ideas concerning applications or "episodes" of photo-interviewing ai'e based on two of 
our attempts to incorporate photo-interviewing with inquiries involving separate program 
innovations. Episode 1 involved an evaluation of an intensive 40-hour hypermedia mini-course for 
higher education faculty and graduate students. The setting of the episode was a computer !ab. 
Episode 2, which is currently in the process of analysis, involved research with high school 
juniors and seniors and took place in a chemistry classroom. The study considered ways that 
matrices can function as systematic organizing structures for concept and simple rule learning. In 
the discussion of the photo-interviewing process which follows, we will use examples from both 
of these inquiries. 

Photo-Interviewing: The Process 

The function of all analysis is to find patterns. To identify patterns, photo-interviewing 
uses photographic tracking and selective responses to "peaks of information", when and where 
they happen. Tnis is a flexible approach which can collect unpredictable as well as scheduled 
behaviors in research and evaluation contexts. As Hall (1974) advises, we approach the 
photographic content with an open strategy in order to respond to their holistic content. At 
present, the strategy we propose consists of a nine-step process. 
1 . Identify major questions and research or evaluation perspective. 
The purpose of this stage is to clarify research or evaluation questions and approaches. 

Example from Episode 1: An evaluation paradigm was negotiated by evaluators with 
stakeholders concerned with the hypermedia workshop. Consistent with initial discussions, this 
negotiation process delineated major questions, sources of evidence, and quality standards for the 
evaluation. Major questions were divided into those affectimg the context, process, and product 
' aspects of the evaluation. 

Context questions dealt with four issues. Why was the class planned? How was the class 
planned in terms of: goals, objectives, and priorities, staffmg, methodologies, prerequisites, and 
assessment? What characterized the setting in terras of unique or common factors as well as 
student and faculty backgrounds? Finally, what resources are available (e.g., hardware, software, 
text, readings, lecture notes, overheads, films, and guest lecturers)? 

Process questions first considered how ongoing events evolved. What were student, 
faculty, and trainer perceptions over time? What was the planned versus real content? What were 
the barriers to success? Also an important process consideration centered on effectiveness. How 
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effective were various events which took place during the mini-course, (i.e., resources, 
processes,handouts, readings, cooperative learning events, cognitive versus behavioralistic 
teaching methods)? Which of Gagne's events of instruction were employed during panicular 
periods? How receptive were the trainees at different junctures? What hindered and helped? What 
were the "light bulb events"? 

Product questions began with how well the predetermined objectives were met. a related 
question considered which consequences (intended and unintended) were actually attained. We 
were also concerned student perceptions regarding the degree of match with early expectations. 
Otlier questions negotiated dealt with the transfer value of content and processes. Product-related 
questions in this particular evaluation were intended to focus especially on prerequisite skills, 
content, processes and sequencing. 

Sources of evidence included photographs of the mini-course as well as panicipant 
comments from photo-interviewing sessions. Many other sources of evidence were collected as 
part of the negotiated paradigm. These include the course syllabus, context inventories, daily 
benchmarks, class size and attendee mortality, faculty vitae, student records, trainee demographics, 
evaluator notes, summative questionnaires, and HyperCard products produced by students. 

Quality standards attempted to match context, process, and product evaluation questions. 
These were numerous, but included depth and breadth measurement; generating infonnation that 
both documents and leads to program improvement; ensuring the transferability of content and 
processes; matching expectations with actual outcomes; communicating well-defmed pre-requisites 
and accommodating diverse disciplines; the attaining both intended and unintended competencies; 
using the events of instruction effectively; efficient sequencing of the course; supplying adequate 
resources; integrating motivation throughout the course. 

Comment: Visual anthropologists and visual sociologists recommend using a less 
stmctured approach toward inquiry when first examining natural phenonema. Program evaluation 
is much more focused in terms of major questions from the very beginning and this is reflected by 
our paradigm. Since photography is negotiated along with otiicr sources of evidence and the aim is 
to lead toward program improvement, we designed our inquiry more deliberately than 
anthropologists or sociologists. 

Z, Prepare the data-gathgring approagh. 

The purpose of this stage is to establish content parameters for die photographs (i.e., to isolate 
topics related to critical incidents). An additional task was to decide on the frequency and extent of 
coverage. 

Example from Episode 2: After a good deal of contextual analysis, the researchers decided 
to concentrate on five topics related to the descriptive research. These topics were describing the 
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strategics used in learning, group work and partnering, feedback among participants, teacher 
contact, and reward-related occurrences. The photographer was free to take as many photographs 
as possible as long as he didn't feel he was being obtrusive. Based on prior experiences, such as 
Episode one, the researchers decided that photographs would be taken at the discretion of the 
photographer rather than at set (predetermined) intervals. 

Comment: From one perspective, it is important that photographers functioning as data 
collectors are extremely open and receptive to the "unanticipated" in the environment in which tliey 
work. On the other hand it is also important that photographers have a perspective and purposely 
capture the "intended" in their inquiry. An excellent photojoumalist, for example, may be assigned 
to cover a fire. The photographer knows she must "capture" critical events or incidents related to 
topics in newspaper or magazine readers' "fire" schemata. The topics are defined, yet the 
photographer is fre'^ to interpret in innumerable ways. 

The following quote is attributed to John Szarkowski, the influential curator of the New 
York's Museum of Modem Aa 

Photography is a system of visual editing. At bottom, it is a matter of surrounding with u 
ft-ame a portion of one's own cone of vision, while standing in the right place at the right 
time. Like chess, or writing, it is a matter of choosing from among given possibilities, but 
in the case of photography the number of possibilities is not finite but infinite. (Sontag, 
1977. p. 192). 

By establishing topics which later will be connected with critical incidents used in photo- 
interviewing, the photographer/data collector becomes both clear about the parameters of the 
episode and responsive to unanticipated phenomena, 

3. Photograph episode. 

The purpose of this stage is to gatiier visual data describing the envirotoient. 

Example from Episode 1: The mini-course was divided into two Friday-Saturday-Sunday 
segments. This was one of our first experiences with foraiativc evaluation using photo- 
' interviewing and we were following the pioneering work of Collier (1967). Thus, we opted during 
the first weekend to shoot at predetermined time intervals (every 20 minutes- one close-up, one 
wide angle). After reviewing the effect of this strategy, wc decided on a more participant-observer 
approach for the second three-day session which responded to critical incidents rather than time. 
Also, during the first three-day session, we used an electronic flash for illumination. During the 
second session, available light was used as a light source. When interviewed later, not one trainee 
(n=27) observed any differences between the first three-day session and the second. Further, most 
participants forgot about being photographed after the first 1/2 hour of the course. 
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Comment: In photographing program participants as part of an inquiry, one is reminded of 
Ziller's (1990) experiments with Self Theory in which subjects were given photographic 
equipment and instructed to photograph various abstract concepts. There is just no separating the 
photographer's self from the photographer's observations. The question is should a photographer 
intent on gathering the most useful data attempt to record "the decisive moment" as the work of 
Cartier-Bresson (1952) exemplifies or, alternately, map the environment as a type of social 
cartography. In responding to this question, first consider the nature of detection. Sometimes the 
camera, as a powerful instrument of description, should be called on to interpret diverse 
perspectives. On other occasions, the camera is needed only to record. Given our research in 
natural settings, we propose, photographers should consciously operate dualistically--both as an 
interpreter and as a recorder. It may even be argued that artificially imposing constraints such as 
shooting only at specified time intervals or using "standard" camera angles or lenses actually 
contributes to a most serious threat to validity: poor visual description. 

4,„0rgani^e and label photographs. 

The purpose of this stage is to organize photographic data in a fashion that will be most useful for 
photo-interviewing. 

Example from Episode 2: Photographs (shot in black and white) were processed and 3 1/2 
X 5 inch proofs were made of each negative. The proof prints were numbered sequentially by roll ( 
e.g., B-31 would be the photo ID number for the 31st exposure on roll B). The classroom clock 
was included in the photographs every so often during the study to provide a temporal reference to 
accompany the chronological succession of images. The time for various shots could also be 
approximated by refering to the stationary video cameras of the classroom that were always 
running during the study. 

After the photos were labeled, the researchers viewed the data as a whole. Photographs 
were screened individually and then, collectively. Data collectors and evaluators mterviewed each 
other about perceptions invoked by the photographs in an effort to balance interpretative with 
recording shots. (Questions and verbal comments of the researchers were noted and referenced to 
photo ID numbers. 

After participants were scheduled for small group interviews (typically triads), photographs 
were clustered to include specific shots of individuals within a particular collaborative learning 
groups being studied as well as generic shots of the whole class. Finally, the photographs were 
organized to correspond to topics complementing the five research questions under study. 

Comment: Whatever the strategy employed, photographs should be assembled in 
preliminary structured units which document a visual narrative. For example, photo events can be 
arranged as coronological experiences, as spatial maps of an environment, or as models. 
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In effect, the analysis begins here. This stage roughly conforms to the beginning of Collier 
& Collier's basic model for analysis (1986, p, 178). At the same time, the stimuli is arranged in a 
way that is intended to provide for efficient interviewing. It may be necessary, for example, to cull 
individual photographs which are clearly redundant. In our first attempts at photo-interviewing, we 
overwhelmed some interviewees by placing too many photographs on the table for their 
consideration. In trying to mentally sort the photographs into logical catagories for discussion, they 
became confused and often "got off-track" in their conversation. After that experience, it was clear 
to us that it is the researchers responsibility to limit the stimuli in a way that both helps subjects' 
recollections and allows some choice among the photographs selected. 

5. Prepare protocol. 

The purpose of this stage is to prepare a protocol (i.e., of verbal and visual questions) for the 
photo-interviewing session. 

Example from Episode 2: These oral interview questions were organized arounr* the 
evaluation paradigm's five major questions which concentrated on learning strategies, group work 
and partnering, feedback, teacher contact, and rewards. In this study, we generated a verbal 
interview schedule which explored each research question and then reorganized the question 
sequence using a loose adaptation of the critical incident technique. 

The protocol or script was very specific in its wording so that the same questions were 
asked of iJl cooperative learning groups. After a brief statement in which subjects were informed 
that the researchers were asking the interview questions to see how well some methods of learning 
work, subjects were asked for general reactions to the five topics. The reader is referred to 
Appendix A for a delineation of the interview schedule. 

The remainder of the protocol, with the exception of directions, were phrased in terms of. 
critical incidents. A section of the interview script, concerned with learning strategies, is shown in 
Appendix B. 

Comment: Flannagan (1954) defines critical incidents as specific reports of observed 
behavior from qualified sources. The critical incident technique is remarkable in that it provides 
open-ended responses which tend to avoid generalizations or opinions. This technique, which 
began as a military training invention, is used in job analysis for those tasks with a flexible or 
undefmable number of "right" ways to behave (Zemke & Kramlinger, 1985). In our judgment, 
this technique encourages participant sharing and the emergence of perceiver diversity without the 
typical oppression of the conforming group. 



ERIC 



6. Pilot interview. 

The purpose of this stage is to go through a "dry run" of the photo-interview. 

?r 17 i 9 BESTCGPYAVAilifilE 



Using Photo-Interviewing 

9 

Example from Episode 2: In this study, as a result of last minute schedule changes at the 
school site, we were able to pilot the photo-interviewing session only with members of the 
research team and the class instructor. As such, we were primarily interested in what could go 
wrong with the photo-interviewing session. Were all photos numbered correctly? Was the video 
equiptment which would record the session properly checked-out? Could any questions in the 
interviewing script be made clearer? Was the interview text clearly associated witii the research 
questions? Could the photo-interview be completed in the time we had available? Several items in 
the interview were revised and some photographs culled as a result of this process. Tliis process 
also resulted in what we referred to as a "bring along checklist" which detailed the procedures, 
timelines, materials, and equiptment necessary for conducting multiple photo-interviews. 

Comment: Piloting the photo-interview with individuals or small groups of the target 
population can minimize implementation problems. While we did not encounter major problems. 
Our efficiency could well have been improved by conducting one-to-one evaluations (Dick & 
C^irey, 1990) with a sample of high school science students. Piloting with target members ensures 
that the interview questions, photographs, and procedures are age and ability appropriate. 
Converse and Presser (1986) recommend ten purposes for "pretesting" the instrument. The first 
four evaluate specific questions for variation, meaning, task difficulty, and respondent interest and 
attention. The last six bear on the instrument as a whole. These weigh the "flow" and naturalness 
of the sections; the order of the questions; skip patterns; timing; respondent interest and overall 
attention; and respondent well-being. 

7. Conduct and videot ape interview. 

The purpose of this stage is to elicit and collect a flow of information regarding the research or 
evaluation questions. 

Example from Episode 2: The photo-interviews closely followed the protocol or script 
discussed in step 5. After the interviewer reviewed the purpose of the study and made assurances 
of confidentiality, four groups of three subjects were interviewed. (There were several absences 
' due to illness.) As this study considered two separate concept training strategies, two photo- 
interview ses.sions were held with each group. Interviews were scheduled a few days after each 
training session. Photographs discussed during this procedure were used to clarify and extend the 
data derived from the oral interviews. 

In each interview, there was one interviewer. A videocamera with a remote microphone 
recorded the interviews for later analysis. A second photo-interviewer was also in the background, 
taking notes and occasionally asking probing questions. 

Comment: During episode 2 we used critical incidents along with photographs. This is 
somewhat different than other researchers using photo-interviewing. Probably this approach is not 
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useful for all situations. Even so, there are three advantages in using questions aimed toward 
critical incidents. First, tliis approach encourages the perspective of "open- inquiry" which is 
particularly valuable in innovative development projects. The subject is encouraged to respond or 
project as they see fit. Second, questions phrased in this way get at facts as perceived by the 
subject-observer. They encourage description, not judgements. The description, however, is open 
to subject projections. Using photographs and anecdotes, the subject has the evidence to describe 
to the interviewer, what in his or her opinion "really happened". Finally, critical incident responses 
arc focused on specific observable occurrences related to the research or evaluation. Tlie critical 
incident technique works with the photographs to assure that subjects' responses spotlight 
scenarios related to reseacher or evaluation questions without cuing subjects to a panicnlar 
viewpoint. 

8. Analyze photographs, field notes, interview notes and videotape . 

The purpose of this stage is to digest visual, auditory, and written records in ways that 
provide insight and information. 

Example from Episode 1: Visual data (still and video) are supersaturated with information. 
This enormous quantity of information presents great opportunity and even greater difficulty in 
approaching analysis. In episode 1, we were faced with making sense of a particular class that 
was, after all, one small pan of a continuing three year project involving very diverse sources of 
evidence. . 

Collier and Collier's (1986) basic model for analysis was adapted for our panicular needs. 
We began by organizing the data somewhat sequentially. We kept a fairly a narrative log of the 
visual chronology, spatial relationships of participants, and instructional content and processes 
during the course. We also had detailed records of the ^hoto-intervicwing process. These we 
gathered along with context and summativc questionnaires, stills, and video to provide a context 
for our inventory. As much as possible, we organized our data into informal categories related to 
our research questions. Our approach at this point was what Collier and Collier refer to as "open 
' viewing". 

Open viewing is an unstructured immersion in the visual record, a repeated viewing of all 
the material that allows you to respond to the images as they are and not simply as you 
expect them to be. ( 1986, p. 181). 

To approach structured analysis, we returned to the paradigm negotiated at the beginning of 
the evaluative effort and especially to the major questions identified in the first step of this process. 
Context, process, and product questions provided a basis for organizing our perceptions. Although 
we did not reduce the questions to the level of microanalysis, we focused the data analysis by 
defining the evaluation questions and issues related to these questions. 
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Tliroughout both the open viewing stage and the focused oiialysis stage, notes were taken 
by the evaluators and much discussion ensued regarding findings. Collier and Collier (1986) have 
commented on the value of these exchanges between researchers. 

"These discussions clarified details, raised important questions, and defined 
conclusions,and the interplay of ideas sharpened our examination of the evidence and the precision 
of our analysis. On a conceptual level these joint viewings were the most productive stages of the 
research. " (p. 177). 

Comment: Finding an effective approach to analysis was (and is) the most difficult pan of 
this process for us. Our approach did not include microanalysis. Our review of the literature 
uncovered three behavior patterns which constitute evidence for microanalysis: proxemics 
(measurements of space); kinesics (messages of body behavior); and choreometrics (patterns of 
behavior through time). Like many evaluators and researchers, we were suuggling to control our 
basic techniques for descriptive analysis without concentrating on a purely "count, measure, and 
quanufy" approach. We felt that quantifying visual information, while being easier to 
proceduralize, would '.end to greatly lengthen the analytical process with no guarantee that the 
results would produce meaningful conclusions. On the other hand, we cenainly do not disdain the 
use of quantitative microanalysis. We only agree with Collier and Collier (1986), that 
microanalysis is a reductionist, clinical technique useful in studying small components of l\a nan 
behavior. Particularly in the case of naturalistic evaluation, microanalysis is not compatible with 
the paradigm. 

9. Summariz e findings. 

The purpose of this step is to make sense of the data analysis and offer conclusions to interested 
parties and stakeholders. 

Example from Episode 1: Evaluation findings were used for internal and external 
purposes. Both uses entailed a two step editing procedure. 

First, the findings were used as part of the internal evaluation of the course, the aim being 
to generate program improvement strategics. A written summary was submitted to project staff. 
This report was accompanied by annotated "generic yet contextually rich" photographs included in 
the body of the text to highlight major findings. Sensitive findings as well as "particular" 
photographs which could be traced back to a particular individual were shared verbally in a 
confidential settiing but not put to paper. This first report attempted to validate findings and 
avoided the inclusion of judgments, /vfterwards, we edited the initial findings, incorporated 
revisions generated by reactions to the first "draft" and added an implications and judgment 
section. 
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The frndings were also used ar. part of the annual documentation to the funding agency 
which supported the development of the hypermedia course. Cleansed by the internal evaluation 
reporting process described in the paragraph above, a "fmal" annual summary was submitted to tlie 
funder. Using an executive summary format, it also incorporated generic photographs (though 
fewer than the internal evaluation repon) in the text to highlight major findings. 

Comment: Overall, it appears that the accuracy and the impact of any summary process 
depends upon the amount of lime available for infonnation feedback to the client, what kind of 
heuristics accompany the data, how the information is "framed", and the form of information 
display. At this juncture, photos seem to help match the client's initial certitude (and more hidden 
agendas) with resultant outcomes. In the case of project directors or funding agency program 
officers who cannot be on-site frequently, photographs helped mitigate the "jaundiced attitude" 
administrators have about typical evaluation reports. Rhetoric was backed up with visible evidence 
of process and product behaviors, a phenomenon very rare in education. Along with other program 
evidence, photos helped lO triangulate the findings, including the presence of client's tunnel vision 
and blind spots (i.e., selective perception). Stakeholders tended to take the evidence "as is", 
demanded additional evidence less, and appeared to be more open to negative feedback and 
suggestions for improvement. One pan of the explanation appears to be die relative ease in 
discerning the differential validity of photographs versus other information sources, especially after 
repeated exposure. 

Evaluation experience indicates that tliis two tiered approach to data sharing is imponant for 
both cognitive and political reasons. The data is most likely to be heard and used by clients if they 
are satisfied with its truth value and utility value. This satisfaction index seems to rise when a 
client feedback about data accuracy is sought before presenting data of a judgmental nature. In 
terms of summarizing and reporting findings to the internal clients (current instructor, project 
director, and future trainers), we found that the use of photos along with narrative text, especially 
that which va:; backed up by quotes from participants, helped clarify the findings. It actually 
appeared tc reduce the amount of feedback time required by the clients as well as enhanced the 
' receptivity n results. This seemed to be especially true when the wrinen and verbal feedback 
included photographs of the clients interacting with participants. 

Issues Rglated tp PhQt<^^^tg^vigwi^g 

Our task as instructional researchers and evaluators concerned the issue of how to use 
photo-interviewing as a tool of inquiry that promoted a shared reality among stakeholders. We 
sought to use photographic evidence as a tool which made episodes oven among the significant 
players. Several additional issues should be considered in implementing photo-interviewing. 
These include cognitive, ethical, and technical issues. 
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1. Cognitive Issues 

The purpose of this section is to explore cognitive dilemmas of photo-interviewing, 
seek meaningful patterns of human behavior, one must first start with human perceptions of 
reality. Specifically, thre^ cognitively complex processes must be examined: 

• the perceptual representation of exchanges ( between information inputs and feedback 
messages) by interviewers and interviiwees; 

• the self-awareness of levels of perceptual repres.intation; and 

• the problem-solving skills of the interpreter regarding t'.e data. 

Perceptual Representation 

Perceptions and judgments of photographs are a function of the interviewer's and 
interviewee's contexts. A variety of dilemmas assault our senses as we try to make meaning of our 
world. By making overt the vulnerable representations of both the photo-interviewer and those 
being interviewed, a synthetic picture of multiple viewpoints and value? can be constructed. 
Through common stimuli (e.g., interviewer questions and photographs), these values can be 
articulated and help focus the study. In addition, these values can serve as judgment criteria for 
decision-making. On the other hand, if the operative values are not known, distortion results in 
terms of what is questioned, how evidence is gathered and how judgments are made. In other 
words, the value of a photograph varies greatly across viewers. One wonders if the repeated 
presentations of photographs to interviewees will confirm Sontag's (1977) speculation that 
repeated exposure to images anesthetizes one's representational capacity. What is the saturation 
point of the viewer to images? Without a doubt, each photograph has multiple meanings and the 
dual capacity to makt reality manageable as well as opaque; continuous as well as discrete; simple 
as well as complex; and revealing as well as concealing. 

Self Awareness 

To come to grips with these operative values and multiple realities, individual*- must 
become more self-aware of their subjectivities. Subjectivity has been defined in the dictionary as 
"the quality of an investigator thai affects the results of observational investigation". If we accept 
Webster's definition, then it is fairly safe to assert that subjectivity operates throughout the entire 
photo-interviewing process. Subjectivity has the capacity to filter, skew, shape, block, transform, 
construe and misconstrue what transpires from the outset of a proi-^ct to its final report (Peshkin, 
1988). If subjective qualities permeate the investigation process at botii conscious and 
unconscious levels, we must seaiT:h for ways to control the potential bias by making these qualities 
more overt. One way to cope with subjective bias is to strive for enhanced validity through 
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multiple perceptions. One could conduct many individual photo-interview sessions and aggregate 
the data OR use group photo-interviewing to permit each perceiver to negotiate his or her subjective 
reality with others in a tangible and immediate manner. By viewing photos of critical incidents in a 
sequence with other participants, the learner and investigator alike can transcend the subjective 
biases resulting from indi\ idually dissociated and temporally "frozen" perceptions. The chances of 
distortion can be further decreased when the data collection method uses more than one sensory 
channel (such as auditory and visual channels involved witn photointerviewing). Finally, the 
shared feedback of the ph.^tos appears to result in significantly better self-awareness than reduced 
feedback treatments (Hannon and Rohrbaugh, 1987). 

Problem-Solving Skills 

Photo-inierviewing resuhs are highly contingent upon the problem-solving and 
interpretative skills of the interviewer and interviewees. Photo-interpretation requires information 
search, and such search is guided by hypotheses which are suggested by sentiotic cues. For 
interviewers and interviewees alike, the skill of generating hypotheses can range from novice to 
expert. Evidence in general problem-solving research suggests that problem-solving is primanly 
schema-driven for the expert and search-driven for the novice G-arkin, 1979; Swellcr and Levine, 
1982; Gilhooly, 1990). This finding has many implications for interpreting interviewee responses 
and evaluating the quality of interviewer question^ . Expert inteipreters should have a better 
memory for new information and a tendency to more clearly perceive the deep structure of the 
problem (i.e., the photograph to be interpreted). Neophyte viewers focus on key words in the 
question and surface structure features of the photograph. 

In fact, the skills involved in photo-interviewing appear to be very similar to clinical 
problem-solving. Elstein, Schulman, and Sprafka's results (1978) suggest medical experts 
generate a few hypotheses within the first few minutes of contact with 'the patient based on 
presented symptoms and signs (reasoning forward). These hypotheses are then tested by deducing 
what other symptoms ought to be present and testing for them (reasoning backwards). Funher, it 
' is suggested that experienced physicians develop "illness scripts" containing prototypical 
informadon about real patients so that the problem-solving process becomes fairly automatic 
(Feltovich and Barrows, 1984; Hobus, Boshuizen and Schmidt, 1989). Since evaluation novices 
lack these real-life experiences, neophytes cannot proceed beyond superficai processing nor 
develop programmadc "sc*ripts". We believe the photo-interviewing strategy assists in training 
novice researchers and evaluators to develop viable scripting skills. Photographs combined with 
verbal data can be overtly manipulated in a variety of contextual and sequential formats, helping 
them to perceive cues on many levels and reason both backwards and forwards. 
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X gthWal Issues 

The purpose of this section is to explore ethical dilemmas of photoiiiterviewing. While the 
ethical implications of interviewing and observation have been studied for many years in 
biomedicine (Fang & EUwein, 1990), visual anthropology (Collier & Collier, 1986) and visual 
sociology (Gold, 1989) , the polemical contexts of educational evaluation and research pose three 
unique questions. 

• How to achieve an accurate representation of events from a research versus an evahjation 
perspective? 

• How can informed consent be truly "informed"? 

• How to increase evaluaior/research vulnerability along with client/respondent 
vulnerability? 

Accurate representation of events: 

In the earlier section on cognitive issues, we discussed the impact of individual perceptions 
on event portrayal. In this section we will attempt to move from the psychology to the ethics of 
photo-interviewing. Achieving accurate portrayal requires the synthesis of interviewer and 
respondent perceptions. Given that researcher and evaluator perceptions are so context bound, 
accuracy demands that the contexts be representatively sampled in terms of time, purpose, 
activity/location, method, controls, and ends. Sampling over time and location is important 
because a program may change from the beginning to the end (Becker, 1978). The purpose of the 
photo-interviewing and specifically how its use is explained to participants can also impact the 
accuracy of results. Accuracy demands sampling both private and public (posed and candid) views 
of participants, which can be perceived as intrusive during the photographic act as well as risky for 
its ultimate reception when shared during the photo-interviewing session. The ethical risks are 
compounded when planned and unplanned activities must be captured; especially when it is 
perceived as "catching someone out". According to Cassell (1982), the "parity of the research 
relationship is based on the perceived power of the investigator and control of the setting... Persons 
' must always be treated as ends in themselves, never merely as means." Evaluators have much less 
control of their respondents, which is more representative of a natural setting. Fang and Ellwein 
(1990) urge consideration of tiie costs and benefits to all participants in terms of financial, 
physical, intellectual, psychological and time factors. 

Informed consent: 

The quest for accuracy is tempered by tiic potential violation of confidentiality assured in 
the "informed consent form" signed by participants. Because of both the public and private nature 
of photogri^hs, it is not possible to conceal the identify of participants nor the setting (Gold, 
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1989). But while the participant may be knowiedgable about the initial concept of the study and 
initially agreeable, the program changes over time, activities, competence, and trust-building 
capacity of the photo-interviewers. Gold emphasizes that this issue is compounded by the 
phenomenon that: a) photo-interviewers and informants are likely to hold different meanings to a 
picture; and b) photos can be easily gathered without participant awareness. The affect triggered 
by a picture can vary "profoundly and unpredictably" during the photography as well as after the 
photos have been developed. Ca.ssell and Wax (1980) advocate substituting tlie one- shot consent 
form into a negotiated "covenant" between the observed and the observers. Tliis relates to Gold's 
coventantal relationship whereby the evaluator and infomiant develop consensus about the framing 
of the photograph, the final print, its use and its interpretation. By asking panicipants to explain 
their interpretations singly and in small groups, insights can be confirmed about the program's 
integral contexts, processes, and products. The high degree of interdependence required to cope 
with this ethical dilemma is much easier to achieve with an evaluation paradigm than an research 
paradigm which posits little or no control by the subjects. And even then, the evaluator must be 
very sensitive to each informant's culture and individual reactions. Accurate portrayal requires an 
evaluator who is not only sensitive and capable of generating meaningful exchanges, he/she must 
be able to read the participants "visual code" over multiple photos and different levels of visibility 
(e.g., oven, opaque, covert). 

Vulnerability 

Who controls the nine phases of photo-interviewing? Where can negotiation play a role? 
How can the openness of both the observed and the observer be assured? Should a narrative 
accompany each photograph? And when can vubcrability become a positive condition rather than 
sometiiing to be avoided at all costs? These questions all have vulnerability and control issues in 
common. Our field studies have shown tiiat the risks of non-negotiation with clients and 
informants is far greater than the reverse condition. Photo-interviewers who share with infomiants 
tiie selection of critical photographs (candid and public) as well as tiieir use in subsequent reports 
and presentations lower the chances of misrepresentation, oppression, and harm. Verbal or written 
narrative which accompanies a photograph further minimizes a photograph being taken out of 
context. Becker (1986) goes on to add the caveat that both photos and corresponding text should 
be approved prior to publication or dissemination. Our experiences led us to develop another 
method for reducing photo-interviewer distortion and oppression-using two photographers to 
capture an event, using the photo-interviewers and sampling of the target group to select 
photographs for inclusion in the interview, and interviewing in teams. 
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The purpose of this section is to consider: 

• recent developments in photographic technology which may affect photographic data 
collection and interviewing; 

• employing still versus motion stimuli for photo-interviewing. 

Recent Developments . 

Among tlie most invigorating aspects of using photo-technologies for data collection and 
photo-interviewing are the enormous changes going on with the technical capabilities of related 
media. The dust has not settled on the answers to ways photography will customarily be viewed in 
the near future. Even so, the handwriting is on the wall for traditional continuous tone images 
printed on photo-sensitive substrates. This is not an argument for technology for its own sake. 
Simply, we make comments here because changes are occuring with the processes with which we 
work. These technology-driven changes, and the long-term reduction in cost per imago which will 
likely accompany them, will encourage greater use of visual media for evaluation and research 
purposes. 

One of the most important recent developments in this regard is the September 1990 
announcement by Eastman Kodak of Photo CD, a format to be available next year that allows the 
average home photographer to take up to 100 photos on a special compact disc and inserts into a 
combination audio OD/Photo CD player and can then be viewed on a television set. 

Another rather recent development is the somewhat recent proliferation of "frame grabbing" 
programs which pull stills or short video segments off of videotape or videodisc. Made possible by 
innovations such as IBM's M-Motion Video card, stills or video may easily be incorporated into 
custom software hypermedia programs such as Toolbox on the IBM or Supercard on the 
Macintosh. The potential for scanning imagery for software programs has been commonplace for 
some time, but these developments greatly simplify the processes associated with capturing and 
inserting researcher or evaluator-generated visuals. Other factors which will push the use of photo- 
related technology are the familiarity with visuals of the 'TV generation", the now everyday use of 
' computer technology by all but the most resistant, and the greatly reduced cost of the large amount 
of computer memory necessary to include graphics in software programs. 

It is reasonable tD expect that research and evaluation models and methods will adapt to the 
unprecedented potential accompanying these technological innovations. Microethnographers are 
already using sophisticated textual analyses to examine speech events recorded in situ and 
transcribed (Levine, 1990) . Likewise, qualitative researchers and evaluators in education and 
related social sciences increasingly will use the photo-related data collection and the computer to 
gain insight into the structure of learning and learning behavior. 
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Still versus motion 

As we have mentioned, it is our preference to gather images for photo-interviewing 
sessions using a still camera. There are three reasons for this approach. First, a still camera is 
small, highly-mobile, and forces the data collector to concentrate on "collectmg" a single, most 
valuable image at a time. A data collector is forced by the medium to look for the visual epitome of 
an occurrence. Second, photographing snippets of information with stills instead of great buckets 
of data with video frees the evaluator or researcher to work with a more manageable amount of 
information. Paring down hours of video for use as stimuli in a photo-interviewing session in 
which even more video will be shot becomes an overpowering, time-consuming task. Third, 
laying out several photographs on a table or pinning images to a wall during a photo-interviewing 
session is a relatively natural way to observe diverse imagery. It is fluently understandable to 
subjects in the way that a still montage is. Subjects invariably home into a single photograph or 
details of a single photograph. It is the same phenomena that is used to identify a suspect in a crime 
or increase the power of a microscope to make details clearer. Humans naturally want to dissect 
information in a visual image. Those who recall David Hemmings' obsession for making the 
strange familiar in Antonioni' s "Blow up" can imagine the disorientation that might be present in 
attempting to really dissect a moving image. Few of us not reared on Nintendo games would be 
capable of it. Perhaps most of us can only truly concentrate on that which is not changing. In 
effect, we become disoriented by the impossibility of totally concentrating on what is changing. 

Conversely, in some cases where time, motion, and sound arc imponant, film or video 
is irreplacable. Video and film, property employed can capture a choreography of interpersonal 
relations in a way that stills usually can not. Artfully shot footage is wonderful for recording the 
tempo of an event. In essence, the arguments against using moving imagery for photo-interviewing 
mean little when motion and sound arc needed for some reason. 

Conclusions and Practical Implications * 

Several preliminary implications, drawn from issues discussed in this paper, a.ic 
communicated below. 

Given the aims of educational evaluation and research vary from that of the pioneers in 
photo-interviewing (i.e., biomedicine, sociology and anthropology), tiie original procedure 
delineated by Collier needed some modification. The nine step process presented in this paper 
seems to address the needs of instructional technology more direcdy. 

Photographs can be used as more than data sources. They can also be used for grounding 
contexts and illustrating concepts across groups of informants and potential "inductees". For 
evaluators and researchers, the photo-interviewing process creates a common and concrete 
reference point for subsequent comments and analysis. Wider use of this methodology will 
certainly be enabled by recent technical developments in photography as well as computer 
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manipulation of qualitative data. One application currently being explored by the authors involves 
blending photo- interviewing with interactive-video disk technology. 

Photo-interviews yielded "richer data" than that obtained from verbal interviewing 
procedures. One reason is thai infonriants tended to examine images more carefully than the written 
or spoken word. Further, it appears photo-interviews can generate data particularly helpful for 
pilot-stage projects where innovation and development rather than maintenance are the priorities. 

Photographs can dissolve the alienating or closed verbal authority of tiie evaluator and 
researcher, likened to the effect of lecture loving professors who in the process of talking 
"squelch" or reject interpretations of students who perceive events differently. By providing a 
common stimulus for all informants, images both solicit significant differences and similarities in 
individual perception which can be statistically analyzed across groups of perceivers. The 
methodology provided a means of "getting inside" the program and its context to describe and 
explain the program and its consequences in terms of participants* realities and meaning systems 
that oral interviewing did not permit. 

To apply this methodology accurately, three cognitively complex processes must be 
examined: the percepmal representation of exchanges between photo-interviewers and 
respondents; assessing and enhancing the self-awareness levels of perceptual representation; and 
systematically building the problem-solving skills of the interpreter regarding potentially 
overwhelming data. 

Photo-interviewing can serve as a research as well as an evaluative strategy given its 
projective nature. Photographs seem to have a life of their own beyond the evaluation or the 
project. For example, tiiose photographs which generated negative perceptions (e.g., viewed as 
inaccurate, disturbing or infuriating by informants) were often indicative of rejection of oven data 
at odds witii tiieir own internal data or belief structures. 

Photo-interviewing does not work in all instances. The different contexts of fcseaich and 
evaluation distinctly effect its viability as illustrated by our two episodes. And ethical qu'.rstions 
need to be repeatedly raised in terms of: how accurate portrayal of events is achieved; how to 
make "informed consent" of participants "informed '; and how to reduce exploitation by increasing 
the inteipersonal vulnerability of the obser\'ed and observers. 

Lastiy, enormous technological innovations, Including radicaily different photo materials 
and hardware, are occuring. Technolog:/ -dri'''sn changes, the long-term reduction in cost per 
image, and improved methods of analysis will encoiTtagc greater use of visual media for evaluation 
and research purposes. 

In conclusion, although it is unusual to end a paei* with a quote, v e feluhe foiiowing 
statement by with John Collier was particularly germane. 
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I recommend a process of photographic traclcing and selective response to 'peaks of 
information' where and when they happen. This is a flexible approach, one that can be 
responsive to unpi^ctable and uncontrollable behavior in social and cultural field settings. 
It is a process in which selectivity is not an immaculate and mechanical exercise, but a 
manifestation of human sensitivity, one that rightfully portrays significant research as an 
accomplishment of human judgement. (Collier, 1979, p. 163). 
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appendix A: Section of a photo-imerview scrip: which emphasizes critical incidents. 

In the next set of questions we are going to lay some pictures of your 
group working on the table. These photos are numbered. Take a minute to 
look at them. 

If any of them remind you of something or are useful to you in making a 
point, use them in any way you want. I'll read off the number when you 
talk about the photograph so that we can tell which photo you were talking 
about later when we look a! the video tapes. 



1. Strategy 

IPl: Look at these photographs. Do any of these photographs remind you of when you 
were having trouble with or else when you starting to feel conifortable [usingjM 
concentration game or coming up with examples] ? {Pause} 

(la) Think of the time that you got stumped [using the concentration game or coming up 
with examples! {Pause} When did this incident happen? What happened? 

(lb) What happened to help you figure out how to use this fusing the concentration game 
or coming up with examples! ? Or did you use another method? {Pause} 

(Ic) Think of the time this Fusing the concentration game or coming DP.yith e.^qmples] was 
the easiest for you to use. {Pause} When did this happen? {Pause} 

(Id) Think of the time you felt comfortable in using this Fusing the concentration game or 
coming up with examples! . {Pause} When did you see the forest through the trees? 
What happened? {Pause} 

(le) Think about some strategies you might normally use in learning material of this type? 
{Pause} Tell me about some occasions when you used your own strategies to learn 
science material. {Pause} 
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